[Misc] Fix `Current vLLM config is not set.` warnings, assert to avoid issues in the future by LucasWilkinson · Pull Request #31747 · vllm-project/vllm

LucasWilkinson · 2026-01-05T21:45:35Z

#30531 and #29575 introduced accesses to get_current_vllm_config on boot that are outside of set_current_vllm_config contexts leading to repeated logs

(EngineCore_DP0 pid=1647618) WARNING 01-05 16:41:49 [vllm.py:1447] Current vLLM config is not set.
(EngineCore_DP0 pid=1647618) INFO 01-05 16:41:49 [scheduler.py:231] Chunked prefill is enabled with max_num_batched_tokens=2048.
(EngineCore_DP0 pid=1647618) INFO 01-05 16:41:49 [vllm.py:635] Disabling NCCL for DP synchronization when using async scheduling.
(EngineCore_DP0 pid=1647618) INFO 01-05 16:41:49 [vllm.py:640] Asynchronous scheduling is enabled.
(EngineCore_DP0 pid=1647618) INFO 01-05 16:41:49 [dp_utils.py:30] Using CPU all reduce to synchronize DP padding between ranks.

when get_current_vllm_config falls back to creating a default config. This also lead to slight config propagation bugs as some custom ops would see the default config instead of the real one.

This PR fixes those accesses and makes it an assert to try to avoid this in the future.

Longer term we should consider something like: #30859

TODO: get CI green with targeted additions of VLLM_ALLOW_DEFAULT_CONFIG

gemini-code-assist

Code Review

This pull request improves the robustness of the vLLM configuration handling by replacing a warning with an assertion when the configuration is not set. This helps to catch potential bugs early. An environment variable VLLM_ALLOW_DEFAULT_CONFIG is introduced to maintain the old behavior for testing purposes, which is a thoughtful addition.

The other changes in the pull request are logical consequences of this stricter configuration check. The use of a lazy dictionary in vllm/model_executor/layers/fused_moe/cpu_fused_moe.py correctly defers the instantiation of CustomOp subclasses until they are needed, avoiding errors during module import. Similarly, caching the GroupedTopk instance in vllm/model_executor/layers/fused_moe/layer.py is a good optimization that also resolves the configuration access issue.

Overall, the changes are well-implemented, improve code quality, and I have no concerns.

mergify · 2026-01-05T21:49:39Z

Hi @LucasWilkinson, the pre-commit checks have failed. Please run:

uv pip install pre-commit
pre-commit install
pre-commit run --all-files

Then, commit the changes and push to your branch.

For future commits, pre-commit will run automatically on changed files before each commit.

Tip

Is mypy or markdownlint failing?

mypy and markdownlint are run differently in CI. If the failure is related to either of these checks, please use the following commands to run them locally:

# For mypy (substitute "3.10" with the failing version if needed)
pre-commit run --hook-stage manual mypy-3.10
# For markdownlint
pre-commit run --hook-stage manual markdownlint

ProExpertProg

Thanks for addressing this!

vllm/config/vllm.py

vllm/model_executor/layers/fused_moe/cpu_fused_moe.py

vllm/model_executor/layers/fused_moe/layer.py

mergify · 2026-01-06T03:07:07Z

Hi @LucasWilkinson, the pre-commit checks have failed. Please run:

uv pip install pre-commit
pre-commit install
pre-commit run --all-files

Then, commit the changes and push to your branch.

For future commits, pre-commit will run automatically on changed files before each commit.

Tip

Is mypy or markdownlint failing?

mypy and markdownlint are run differently in CI. If the failure is related to either of these checks, please use the following commands to run them locally:

# For mypy (substitute "3.10" with the failing version if needed)
pre-commit run --hook-stage manual mypy-3.10
# For markdownlint
pre-commit run --hook-stage manual markdownlint

mergify · 2026-01-06T04:16:38Z

Hi @LucasWilkinson, the pre-commit checks have failed. Please run:

uv pip install pre-commit
pre-commit install
pre-commit run --all-files

Then, commit the changes and push to your branch.

For future commits, pre-commit will run automatically on changed files before each commit.

Tip

Is mypy or markdownlint failing?

mypy and markdownlint are run differently in CI. If the failure is related to either of these checks, please use the following commands to run them locally:

# For mypy (substitute "3.10" with the failing version if needed)
pre-commit run --hook-stage manual mypy-3.10
# For markdownlint
pre-commit run --hook-stage manual markdownlint

ProExpertProg

Just a nit for the error message, thanks for this cleanup!

vllm/config/vllm.py

mergify · 2026-01-06T16:54:49Z

Hi @LucasWilkinson, the pre-commit checks have failed. Please run:

uv pip install pre-commit
pre-commit install
pre-commit run --all-files

Then, commit the changes and push to your branch.

For future commits, pre-commit will run automatically on changed files before each commit.

Tip

Is mypy or markdownlint failing?

mypy and markdownlint are run differently in CI. If the failure is related to either of these checks, please use the following commands to run them locally:

# For mypy (substitute "3.10" with the failing version if needed)
pre-commit run --hook-stage manual mypy-3.10
# For markdownlint
pre-commit run --hook-stage manual markdownlint

vllm/config/vllm.py

Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>

…d issues in the future (vllm-project#31747) Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com> Signed-off-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com> Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com>

nvpohanh · 2026-01-09T07:40:31Z

Thanks for fixing this! Really appreciate it

…d issues in the future (vllm-project#31747) Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com> Signed-off-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com> Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com>

…d issues in the future (vllm-project#31747) Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com> Signed-off-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com> Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com> Signed-off-by: dsuhinin <suhinin.dmitriy@gmail.com>

…d issues in the future (vllm-project#31747) Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com> Signed-off-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com> Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com>

vLLM v0.14.0 (vllm-project/vllm#31747) changed get_current_vllm_config() from a warning to a hard AssertionError when called outside a set_current_vllm_config() context. This broke all vLLM collectors on v0.17.0 (1793 errors in pipeline 47590887). Three fixes: - utils.py: wrap ensure_model_parallel_initialized() in config context inside setup_distributed() (fixes 1559 errors across gemm, mla, dsa) - collect_moe.py: pass dp_size=1 to FusedMoE() constructor (fixes 32 mxfp4 errors where vLLM tried to query uninitialized DP group) - collect_moe.py: add is_gated_activation=True to prepare_static_weights_for_trtllm_fp4_moe() (fixes 202 nvfp4 errors from new required arg in vLLM v0.17.0) Also wraps all MoE benchmark runs in set_current_vllm_config() context, adds _nvfp4_available gate, and bumps __compat__ to vllm>=0.14.0. Signed-off-by: Simone Chen <simonec@nvidia.com>

…0+) (#688) * fix: vLLM >= 0.14.0 collector compatibility for set_current_vllm_config vLLM v0.14.0 (vllm-project/vllm#31747) changed get_current_vllm_config() from a warning to a hard AssertionError when called outside a set_current_vllm_config() context. This broke all vLLM collectors on v0.17.0 (1793 errors in pipeline 47590887). Three fixes: - utils.py: wrap ensure_model_parallel_initialized() in config context inside setup_distributed() (fixes 1559 errors across gemm, mla, dsa) - collect_moe.py: pass dp_size=1 to FusedMoE() constructor (fixes 32 mxfp4 errors where vLLM tried to query uninitialized DP group) - collect_moe.py: add is_gated_activation=True to prepare_static_weights_for_trtllm_fp4_moe() (fixes 202 nvfp4 errors from new required arg in vLLM v0.17.0) Also wraps all MoE benchmark runs in set_current_vllm_config() context, adds _nvfp4_available gate, and bumps __compat__ to vllm>=0.14.0. Signed-off-by: Simone Chen <simonec@nvidia.com> * feat: add nvidia/MiniMax-M2.5-NVFP4 to collector MoE test cases Signed-off-by: Simone Chen <simonec@nvidia.com> * fix: wrap vLLM all_reduce collector initialize_model_parallel in config context collect_all_reduce.py has its own setup_vllm_distributed() that calls initialize_model_parallel() without set_current_vllm_config() context, causing AssertionError on all ranks with vLLM >= 0.14.0. Signed-off-by: Simone Chen <simonec@nvidia.com> --------- Signed-off-by: Simone Chen <simonec@nvidia.com>

* fix: vLLM >= 0.14.0 collector compatibility for set_current_vllm_config vLLM v0.14.0 (vllm-project/vllm#31747) changed get_current_vllm_config() from a warning to a hard AssertionError when called outside a set_current_vllm_config() context. This broke all vLLM collectors on v0.17.0 (1793 errors in pipeline 47590887). Three fixes: - utils.py: wrap ensure_model_parallel_initialized() in config context inside setup_distributed() (fixes 1559 errors across gemm, mla, dsa) - collect_moe.py: pass dp_size=1 to FusedMoE() constructor (fixes 32 mxfp4 errors where vLLM tried to query uninitialized DP group) - collect_moe.py: add is_gated_activation=True to prepare_static_weights_for_trtllm_fp4_moe() (fixes 202 nvfp4 errors from new required arg in vLLM v0.17.0) Also wraps all MoE benchmark runs in set_current_vllm_config() context, adds _nvfp4_available gate, and bumps __compat__ to vllm>=0.14.0. Signed-off-by: Simone Chen <simonec@nvidia.com> * feat: add nvidia/MiniMax-M2.5-NVFP4 to collector MoE test cases Signed-off-by: Simone Chen <simonec@nvidia.com> * fix: wrap vLLM all_reduce collector initialize_model_parallel in config context collect_all_reduce.py has its own setup_vllm_distributed() that calls initialize_model_parallel() without set_current_vllm_config() context, causing AssertionError on all ranks with vLLM >= 0.14.0. Signed-off-by: Simone Chen <simonec@nvidia.com> * fix: resolve MLA module collector model paths locally to avoid HF downloads collect_mla_module.py passes model names (e.g. "deepseek-ai/DeepSeek-V3") to vLLM's ModelConfig which calls AutoConfig.from_pretrained(), requiring network access to HuggingFace Hub. In CI containers without internet this causes OSError for all test cases (4210 errors in pipeline 292175043). The HF configs already exist in src/aiconfigurator/model_configs/. Add _resolve_model_path() that creates a temp directory with a symlink to the local config.json, so vLLM loads from disk instead of downloading. Signed-off-by: Simone Chen <simonec@nvidia.com> --------- Signed-off-by: Simone Chen <simonec@nvidia.com>

LucasWilkinson requested review from ProExpertProg, WoosukKwon, hmellor, houseroad, mgoin, pavanimajety, robertgshaw2-redhat, tlrmchlsmth, yewentao256 and youkaichao as code owners January 5, 2026 21:45

mergify bot added the cpu Related to CPU backends label Jan 5, 2026

gemini-code-assist bot reviewed Jan 5, 2026

View reviewed changes

ProExpertProg reviewed Jan 5, 2026

View reviewed changes

vllm/config/vllm.py Outdated Show resolved Hide resolved

vllm/model_executor/layers/fused_moe/cpu_fused_moe.py Show resolved Hide resolved

vllm/model_executor/layers/fused_moe/layer.py Show resolved Hide resolved

LucasWilkinson added the ready-run-all-tests Trigger CI with all tests for wide-ranging PRs label Jan 5, 2026

LucasWilkinson requested review from DarkLight1337, jeejeelee and ywang96 as code owners January 6, 2026 00:42

mergify bot added multi-modality Related to multi-modality (#4194) nvidia v1 labels Jan 6, 2026

github-project-automation bot added this to NVIDIA Jan 6, 2026

ProExpertProg mentioned this pull request Jan 6, 2026

[Bug]: DeepGEMM MoE generating tons of warnings #30571

Closed

1 task

ProExpertProg linked an issue Jan 6, 2026 that may be closed by this pull request

[Bug]: set_current_vllm_config() is only done during the initialization stage but not the runtime stage #30859

Closed

1 task

ProExpertProg approved these changes Jan 6, 2026

View reviewed changes

vllm/config/vllm.py Outdated Show resolved Hide resolved

github-project-automation bot moved this to Ready in NVIDIA Jan 6, 2026

ProExpertProg approved these changes Jan 6, 2026

View reviewed changes

vllm/config/vllm.py Outdated Show resolved Hide resolved

LucasWilkinson added 8 commits January 8, 2026 15:45

fix tests

6f6503e

Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>

fixes

d36204a

Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>

fixes

4fe4001

Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>

wip

818dcbf

Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>

fix tests

a7cdeb8

Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>

fixes

59b675c

Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>

fix

cca8fa7

Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>

fix

54b1339

Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>

LucasWilkinson force-pushed the lwilkinson/fix-warnings branch from ff497f1 to 54b1339 Compare January 8, 2026 15:45

LucasWilkinson requested a review from ApostaC as a code owner January 8, 2026 15:45

mergify bot added the kv-connector label Jan 8, 2026

fixes

2cb7e28

Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>

simon-mo disabled auto-merge January 8, 2026 23:20

simon-mo merged commit 6cdf015 into vllm-project:main Jan 8, 2026
138 of 140 checks passed

github-project-automation bot moved this from Ready to Done in NVIDIA Jan 8, 2026

kzwrime mentioned this pull request Jan 15, 2026

[Bug]: _CPU_MOE_ACT in cpu_fused_moe_torch cause AssertionError: Current vLLM config is not set #32368

Closed

1 task

tlrmchlsmth mentioned this pull request Feb 10, 2026

[Misc] Add pre-commit hook to catch boolean ops in with-statements #34271

Merged

4 tasks

inkcherry mentioned this pull request Feb 19, 2026

[CI][AMD][BugFix][P/D] Add default_vllm_config to test_moriio_connector.py so tests pass #33739

Merged

2 tasks

simone-chen mentioned this pull request Apr 3, 2026

fix: vLLM collector compatibility for set_current_vllm_config (v0.14.0+) ai-dynamo/aiconfigurator#688

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[Misc] Fix `Current vLLM config is not set.` warnings, assert to avoid issues in the future#31747

[Misc] Fix `Current vLLM config is not set.` warnings, assert to avoid issues in the future#31747
simon-mo merged 26 commits intovllm-project:mainfrom
neuralmagic:lwilkinson/fix-warnings

LucasWilkinson commented Jan 5, 2026 •

edited by github-actions bot

Loading

Uh oh!

gemini-code-assist bot left a comment

Uh oh!

mergify bot commented Jan 5, 2026

Uh oh!

ProExpertProg left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

mergify bot commented Jan 6, 2026

Uh oh!

mergify bot commented Jan 6, 2026

Uh oh!

ProExpertProg left a comment

Uh oh!

Uh oh!

mergify bot commented Jan 6, 2026

Uh oh!

Uh oh!

Uh oh!

nvpohanh commented Jan 9, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Uh oh!

Conversation

LucasWilkinson commented Jan 5, 2026 • edited by github-actions bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

gemini-code-assist bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

mergify bot commented Jan 5, 2026

Uh oh!

ProExpertProg left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

mergify bot commented Jan 6, 2026

Uh oh!

mergify bot commented Jan 6, 2026

Uh oh!

ProExpertProg left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

mergify bot commented Jan 6, 2026

Uh oh!

Uh oh!

Uh oh!

nvpohanh commented Jan 9, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

LucasWilkinson commented Jan 5, 2026 •

edited by github-actions bot

Loading